Stochastic realization theory for exact and approximate multiscale models

The thesis provides a detailed analysis of the independence structure possessed by multiscale models and demonstrates that such an analysis provides important insight into the multiscale stochastic realization problem. Multiscale models constitute a broad class of probabilistic models which includes the well-known subclass of multiscale autoregressive (MAR) models. MAR models have proven useful in a variety of different application areas, due to the fact that they provide a rich set of tools for various signal processing tasks. In order to use these tools, however, a MAR or multiscale model must first be constructed to provide an accurate probabilistic description of the particular application at hand. This thesis addresses this issue of multiscale model identification or realization. Previous work in the area of MAR model identification has focused on developing algorithms which decorrelate certain subsets of random vectors in an effort to design an accurate model. In this thesis, we develop a set-theoretic and graph-theoretic framework for better understanding these types of realization algorithms and for the purpose of designing new such algorithms. The benefit of the framework developed here is that it separates the realization problem into two understandable parts – a dichotomy which helps to clarify the relationship between the exact realization problem, where a multiscale model is designed to exactly satisfy a probabilistic constraint, and the approximate realization problem, where the constraint is only approximately satisfied. The first part of our study focuses on developing a better understanding of the independence structure exhibited by multiscale models. As a result of this study, we are able to suggest a number of different sequential procedures for realizing exact multiscale models. The second part of our study focuses on approximate realization, where we define a relaxed version of the exact multiscale realization problem. We show that many of the ideas developed for the exact realization problem may be used to better understand the approximate realization problem and to develop algorithms for solving it. In particular, we propose an iterative procedure for solving the approximate realization problem, and we show that the parameterized version of this procedure is equivalent to the wellknown EM algorithm. Finally, a specific algorithm is developed for realizing a multiscale model which matches the statistics of a Gaussian random process. Thesis Supervisor: Alan S. Willsky Professor of Electrical Engineering and Computer Science